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Abstract Body 



Background / Context: The importance of a well-trained scientific workforce to the nation’s 
economic vitality cannot be overstated. We rely on scientists and engineers to ensure our 
national security, to solve our most critical problems, and to increase our general standard of 
living (Hira, 2010). Despite the essential role these specialized workers play, policy makers, 
educators, and business leaders are deeply concerned about the quantity and quality of American 
youth in line to adopt these positions. In terms of quantity, the percentage of undergraduates 
pursuing degrees in engineering, physical science, and math have remained stagnant, and the 
percentage pursuing computer science has declined (National Science Board, 2010). In terms of 
quality, American youth are ill-prepared relative to their international counterparts. For example, 
15 year-olds in the United States rank 16 out of 26 countries in science literacy and 19 out of 26 
countries in mathematics literacy (National Science Board, 2010). These statistics do not bode 
well for a young workforce entering a highly competitive and global economy. 

Due to these concerns regarding quantity and quality, there has been an increased pressure on the 
education system to take a more proactive approach in preparing youth for careers in STEM 
fields. In response, a number of federal policy initiatives have been enacted with specific 
provisions for the enhancement of the STEM curriculum in schools. For example, the Carl D. 
Perkins Career and Technical Education Improvement Act of 2006 provides funds for the 
development of STEM curricula for both college and non-college bound youth, and the America 
COMPETES Act of 2007 supports programs that increase the number of teachers qualified to 
teach STEM in high school. Despite this push at the federal level, education in the United States 
is largely designed and administered at the local level, and consequently, the adoption and 
adaptation of STEM support mechanisms varies considerably across schools, districts, and states. 

In general, there are two types of STEM courses: academic and applied. Academic STEM 
courses include math and science courses that comprise the traditional academic curriculum. 
These include courses such as algebra, geometry, calculus, biology, chemistry, and physics. 
Academic courses are typically taught from a theoretical approach that stresses observation, 
identification, documentation, and computation. Concepts are taught in the context of the 
classroom, where knowledge is largely for knowledge sake, or at the very least, for an exam. 

Applied STEM courses, on the other hand, stress the application of academic concepts to “real 
world” job experiences while incorporating quantitative skills, logic, and problem solving. By 
design, applied courses impart skills and knowledge that have direct “currency” to the daily 
challenges and problems students will face should they pursue a STEM career. There are two 
main strands of applied STEM in the high school curriculum: computer/information sciences 
courses and engineering technologies courses.^ Computer/information sciences include courses 
such as introduction to computer science, C++ programming, visual basic programming, and 
data processing. Engineering technologies courses include surveying, electrical engineering, 
structural engineering, and computer-assisted design/drafting. Nationally, among the class of 
2005, 19 percent of students had earned course credit in computer/information science and 12 
percent had earned course credit in engineering technologies (Hudson & Laird, 2009). 



^ For ease of expression, we use the expression “applied STEM courses” to refer to both computer/information 
science courses and engineering technologies courses throughout. 



SREE Spring 2012 Conference Abstract Template 



A-1 




While a large number of studies demonstrate that academic STEM courses are associated with 
gains in learning (Bozick & Ingels, 2008) as well as favorable postsecondary education outcomes 
(Tyson, Lee, Borman, & Hanson, 2007), and favorable labor market outcomes (Gamoran, 1998), 
it is unknown whether the same holds true for applied STEM courses. Some research on 
occupational coursetaking more broadly - of which applied STEM is only one component - 
finds that these courses in the aggregate (which also include business, manufacturing, etc.) have 
little bearing on learning (Agodini, 2001; Rasinski & Pedlow, 1998), but may improve 
employment prospects (Ainsworth & Roscigno, 2005; Bishop & Mane, 2004). However, to our 
knowledge, there is no research that explicitly examines engineering technologies and 
computer/information science courses apart from the range of occupational courses in student 
curriculum. Therefore, our research will produce new knowledge on the efficacy of applied 
STEM courses in the larger context of math and science coursetaking. 

Purpose / Objective / Research Question / Focus of Study: Academic math and science 
courses have been long shown to increase learning and educational attainment (Bozick & Ingels 
2008; Tyson et ah, 2007), but are they sufficient on their own to prepare youth for the challenges 
and rigor of the STEM workforce? Or, are there distinctive benefits to complementing these 
traditional academic courses with applied ones? Answers to these questions are particularly 
critical as schools try to balance the competing demands of providing youth with the applied 
skills and knowledge to thrive in the adult labor force while at the same time ensuring they are 
meeting high standards of academic competency that are required for college. With respect to the 
latter, the emphasis on standardized testing has exacerbated these concerns, and in some cases, 
test preparation and an over emphasis on academic subjects has “crowded out” other components 
of the curriculum (Nichols & Berliner, 2007). With these competing demands and faced with 
limited resources, education policy makers must make informed choices regarding how to best 
structure their curriculum to meet the needs of a diverse student population. To this end, our 
proposed project will address three research questions: 

1 . What applied STEM courses are most commonly taken by high school students? 

2. To what extent are high school students taking both academic and applied STEM 
courses? 

3. Do applied STEM courses in high school improve achievement in math? 

Setting: Public and Private Secondary Schools in the United States 

Population / Participants / Subjects: For this project, we propose to analyze the Education 
Longitudinal Study of 2002 (ELS:2002), a nationally representative study of lOth-graders in 
2002 sponsored by the National Center for Education Statistics (NCES). ELS:2002 was collected 
via a two-stage sampling procedure. In the first stage, a sample of 750 high schools, both public 
and private, were selected with probabilities proportional to their size. In the second stage, 
approximately 30 students were randomly sampled from each school on the condition that they 
were in the 10th grade in the spring term of the 2001-02 school year, yielding a sample of 
approximately 15,360 students. These 10**’ graders were interviewed about their school and home 
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experiences and they were administered a mathematics assessment. Sample members were re- 
interviewed in the spring of 2004 when most were seniors, and again in the spring of 2006, when 
most were two years out of high school.^ Additionally, their high school transcripts were 
collected. For this paper, we will analyze the 14,290 sample members who have full transcript 
information - a necessary condition for us to assess coursetaking in high school. 

Intervention / Program / Practice: The key “intervention” studied in this paper is academic 
STEM courses and applied STEM courses taken during high school. Academic courses include 
math and science and applied courses include computer/information sciences and engineering 
technologies. These classifications are taken from the Secondary School Taxonomy, which 
organizes all high school courses recorded on students’ transcripts into four distinct curricula; 
academic, career and technical education (CTE), enrichment/other, and special education. Math 
and science are “traditional” subjects that - along with English, social studies, fine arts, and non- 
English language - comprise the academic curriculum. Computer/information sciences and 
engineering technologies, on the other hand, have a distinct occupational focus and are part of 
the CTE curriculum. This taxonomy is mutually exclusive such that courses classified as 
academic are not also classified as CTE and vice versa. 

Research Design: Our analysis will be comprised of two parts: a descriptive overview of the 
STEM coursetaking landscape in high school (to answer research questions 1 & 2) and an 
econometric test of the efficacy of applied math and science coursetaking in promoting 
advancement toward STEM careers (to answer research question 3). The descriptive overview 
will document the types of math and science courses that students take, when in high school they 
take them, and in what combination. See Tables 1 and 2 in the appendices as examples of the 
format of our descriptive overview, (please insert tables 1 and 2 here) 

To address question 3, we will apply multivariate, quasi-experimental techniques to discern 
whether academic courses, applied courses, or combinations of the two support math learning. 
We operationalize math learning using scores on a standardized math assessment, administered 
in both the base-year (10**’ grade) and first follow-up (12* grade) surveys. These assessments 
were designed to maximize the accuracy of measurement that could be achieved in a limited 
amount of testing time while minimizing floor and ceiling effects, by matching sets of test 
questions to initial estimates of students’ achievement. For this analysis, six measures of 
mathematics achievement based on performance on this test are used: a number-right score and 
five proficiency probability scores. The number-right score is an overall measure of 
mathematical knowledge and indicates the number of items answered correctly. A proficiency 
probability score is a criterion-referenced score indicating mastery of specific skills and 
knowledge. Using these scores as outcomes, we will estimate the following model: 



Tikst - Po + piAPEit + P 2 ACAit + Psht + P4Fit + psSt + PeRt + P7Yik(t-p + ytkst 

where Y is standardized math performance for student i in school k in state s at time t as the 
dependent variable on the left-hand side of the equation and at time t-1 as the lagged measure of 
achievement on the right-hand side of the equation. Empirically, the sets of independent 



^ Sample members were followed regardless of whether they changed schools, dropped out, or were held back a 
grade. 
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variables, described by the education production function, would be estimated as follows. At this 
student level, APL is a binary indicator, designating whether a student has taken applied 
coursework. To account for the extent of academic coursework a student has taken, AC A is a 
vector of binary variables indicating the highest level of mathematics course taken. Other sets of 
student-level variables include: I, a vector of a student’s characteristics and F family 
characteristics. At the school level, the model assigns the following inputs: S are school-specific 
characteristics and R are regional indicators. Finally, the error term y includes all unobserved 
determinants of achievement. It is also in this error term that state fixed effects are identified: 
they control for the common influences of states by capturing differences across each state. 

To account for unobserved heterogeneity at the school level, we employ school fixed effects as a 
second model in our analyses. A reason for school fixed effects arises because there are potential 
unobserved school factors that may be influencing both student selection of coursework as well 
as testing performance. For instance, it might be hypothesized that being part of an academically- 
involved school environment might simultaneously be correlated with selection of academic over 
applied coursework as well as math test performance. Thus, this unobserved school environment 
may cause an overestimation of the effect of academic STEM coursework and underestimate the 
effect of applied coursework. 

Data Collection and Analysis: This project makes use of data that have already been collected 
by NCES. In all of our analyses that require tests of significance, we will adjust our standard 
errors using Taylor-series linearization methods to account for the clustered and stratified 
sampling design of ELS:2002. We will also use probability weights to adjust our estimates for 
sampling and item non-response. Also, we will examine the extent of missing data on critical 
items in our analysis, and explore imputation options should missing data threaten to bias our 
estimates. 

Findings / Results: This analysis will be supported by a National Science Foundation grant that 
was recently awarded to the authors. The full paper, with results, will be ready in time for the 
SREE 2012 conference. 

Conclusions: Our study will help contribute to our knowledge of the STEM curriculum in high 
school in two distinct ways. First, this study will be the first to explore applied STEM courses 
taken by students. Currently, there are no national estimates on the types, timing, and prevalence 
of applied STEM courses taken by U.S. students. Second, this study will be the first to jointly 
evaluate the efficacy of both academic and applied STEM courses. Although a number of 
national studies have examined the importance of math and science courses for academic 
outcomes, none have looked at whether applied STEM courses yield the same results and none 
have looked at how taking both academic and applied STEM courses support advancement 
toward STEM careers. This is particularly important as students are increasingly taking both 
types of courses. When full results are available, our conclusions will further explicate how these 
two contributions contribute to a broader policy discussion on the structure of the high school 
curriculum in a changing economic environment. 
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Appendix B, Tables and Figures 



Table 1. Percentage Earnings Credits in Engineering Technologies Courses 



Percent Among Credit Earner s 



Percent 


Earned 


Earned 


Earned 


Earned 


earning 


Credit in 


Credit in 


Credit in 


Credit in 


credits 


9th Grade 


10th Grade 


1 1th Grade 


12th Grade 



Engineering Technologies (all) 
Surveying 

Electrical Engineering 
Computer Assisted 
Design/Drafting 
Course title 
Course title 
etc. 



Table 2. Percentage Earnings Credits in Engineering Technologies Courses for Each Eevel of the 
Math Pipeline 

Eevel Eevel Eevel Eevel Eevel Eevel Eevel Eevel 

1 2 3 4 5 6 7 8 

Engineering Technologies (all) 

Surveying 

Electrical Engineering 
Computer Assisted 
Design/Drafting 
Course title 
Course title 
etc. 
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